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Approaches to Individualization: 
Toward a typology of Innovative Educational Practices 

Gary J. Coles 
American Institutes for Research 
Introduction 

During the 1970-71 school year, the American Institutes for Research 
(AIR) began full implementation of a U. S. Office of Education (USOE) 
sponsored longitudinal study of intensive^ innovative education practices. 
The purpose of tMs project was to examine over a three-year period the 
relative effectiveness of various educational experiences or treatments. 
It was anticipated that these data would provide a valuable source of in- 
formation that could be used in designing new educational programs and re- 
search efforts in the years ahead. This paper will discuss the general 
design and research goals of this study, the Longitudinal Study of Educa- 
tional Practices (Project LONGSTEP) and then review the methodology used 

(1) to gather the study's treatment data and (2) to create treatment types 
representing the approaches to individualized instruction present in par- 
ticipating schools. 

Introduction to Project LONGSTEP 
The objective of Project LONGSTEP was to determine as conq>rehensively 
as possible over a three-year period (1) the relationships between selected 
innovative education treatments and student achievement and attitudes, and 

(2) the components of these treatments which have had the greatest itqjact 
on student outcomes. To realize this goal, the general design of the study 
involved the collection of three major categories of data each school year: 
(1) educational treatment data . conq;>o8ed of variables which measure the edu- 
cational environment of which the student is a mecDber, including 



(a) characteristics of the educational programCs) in which each student 
participates and (b) characteristics of each student's teacherCs); 
(2) st udent characteristics (e.g., eex, socioeconomic status) brought by 
the student into his educational environment; and (3) student outcomes , 
including both cognitive and attitudinal outcome variables* Student data 
were obtained from specially designed student questionnaires yielding back- 
ground information and attitudinal outcome measures and from standardized 
achievement tests yielding cognitive outcome variables. Teacher data were 
obtained from a questionnaire completed by each teacher. Educational 
treatments were documented by AIR staff via information gathered from 
interviews with principals and teachers, from classroom observations ano 
from existing documentatit a of the treatment. Lastly, information Iden- 
tifying each student's teachers (by subject) was obtained and used to 
relate specific treatment and teacher data to individual students. 

Since schools participated in Project LONGSTEP on a purely voluntary 
basis, it was not feasible to randomly assign students or student groups 
to treatment conditions; nor was it possible for AIR to systematically 
vary the treatments present in any given school. Rather, variation among 
the primary independent variables of th study was achieved through the 
selection of existing school programs. Thus, the schools invited to par- 
ticipate voluntarily in the study were chosen because, as a group, they 
represented a range of innovative practices and because they also varied 
with respect to other educationally relevant characteristics (e.g., socio- 
economic level). Approximately 30,000 students, 1,500 teachers and 80 
schools in 13 school districts located throughout the United States even- 
tually participated in Project LONGSTEP during its three years of data 
collection (1970-71, 1971-72 and 1972-73). 



Treatment Data Collection Methodology 
Educational programs are composed of numerous related and unrelated 
processes or treatment components. For this reason, it vms necessary no 
develop a system by which complex <dducational experiences present in the 
diverse sample of participating schools could be described and quantified 
with respect to specific observable educational characteristics. An Edu- 
cational Experience Analysis Guide (EdExAG) was developed for this purpose. 
This guide provided the conceptual scheme and the practical questions and 
alternative responses that AIR project personnel could use in documenting 
the specific experiences of the students participating in the study. This 
guide, consisting of more than 80 items, was designed so that for each item 
it was possible to code the basic components or elements underlying a treat- 
ment on a continuum from "traditional" to "Innovative." Quantitative mea- 
sures of educational "treatment" derived from this guide could then be 
associated \rith participating etudents, ragardless of the schools which 
they attended. 

The data collection unit for educational treatment data was a flexibly 
defined entity called an Edvcational Experience Analysis Guide group or 
EdExAG group. Different EdExAG groups were identified within a school 
when an AIR site visitor could differentiate among the treatments received 
by different groups of students at a school by means of items found on the 
EdExAG. Defined in this manner, EdExAG groups could include as few students 
as those within one teacher's class for one subject matter area or as many 
stulents as those in all grades within a school for all subject matter areas 
of instruction. EdExAG groups, then, were created in response to treatment 
variations that existed within a school and within a grade and may be more 
or less viewed as an efficient but group- level approximation to the 
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documentation of each Individual student *s educational treatment. 

During the 1970-71 school year» this treatment documentation method- 
ology led to the Identification of 141 such EdExAG groups* while 167 and 
228 groups were isolated during 1971-72 and 1972-73 respectively. These 
groups* then» represented all of the different kinds of educational treat- 
ments that we»e provided by the schools to which students participating In 
Project LONGSTEP were exposed during each year of the project. 

Measures of Educational Experiences 

Prior to Initiating analyses utilizing the EdEx/VG data, senior project 
staff who were familiar with the sites participating in the study and with 
educational program organization in general critically Inspected each EdExAG 
item. Those items or combinations of items (i.e., tentative multi-item 
scales) were Identified that would quantify what were judged to be the basic 
educational characteristics underlying the educational practices and pro- 
cedures included in the study. Item and scale analysis procedures were 
then undertaken so that the measurement properties of the EdExAG items and 
the tentative scales might be examined. 

The first step in the item/scale analyses of the EdExAG was to rank 
order the alternatives for each of the items on an a priori scale from 
lesser to greater innovatlveness. Senior-level project staff then examined 
the intercorrelatlons among the items to confirm that scale items were more 
highly related to other items in the same scale than to items measuring 
different constructs. Final scale content was determined by considering 
these empirical relationships and by reaffirming that the items were mea- 
suring what were judged to be the same educational construct on a priori 
grounds. Last, the internal consistency of each multi-item scale was com- 
puted to provide some estimate of the reliability of the measurement 
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provided by ttach scale. Cronbach*s (1951) coefficient alpha (a) was used 
for this purpose. Coefficients for the multi-Item scales ranged from .54 
to .82. 

The Concept of the Treatment Group 

For purposes of collecting treatment Information during site visits 
to participating schools, educational treatments could not be defined In-* 
dependently of the school at which the Information was obtained. A site 
visitor, for example, could only document educational experiences of the 
students In the participating school he or she was visiting, thereby Iden-^ 
tlfylng what have been called EdExAG groups. The data gathered at this 
stage, then, differentiated among groups of participating students within 
the same sdiool who were exposed to different educational practices* De** 
fined in this limited manner, a treatment could Include, at most, all the 
students in a single school, provided that all these students had similar 
educat lonal ei-pHr iences . 

It was originally thought, however, that EdExAG groups defined by 
this approach would not represent educationally unique configurations of 
practices. Some of the small, nonsystematic differences describing the 
EdExAG groups in each school could be educationally trivial or represent 
inaccuracies in documentation. Similarly, groups characterized by sup-> 
posedly identical educational practices may not represent exactly the 
same treatment due to the lack of discriminability of the items on the 
EdExAG. 

Considerations such as these, coupled with a concern that the project* 
analyses should focus on a limited number of educationally meaningful and 
identifiable student groups, suggested that students should be combined 
into larger analytic units based upon the similarity of their educational 



experiences. Such a group was called a treatment group or treatment unit. 
Examination of achievement and attitude differences among such groups 
would then provide insights concerning the efficacy of the combinations 
of practices they Involved. 

Cluster Analysis Methodology 

Combining EdExAG groups into lar^ analytic units representing dif» 
ferent "types" of educational approaches was accomplished by means of a 
cluster analysis procedure based upon Q-type factor analysis. Separate 
analyses were conducted for the data collected each year and for language 
arts (n • 92, 1971-72; n « 103, 1972-73) and mathematics (n » 89, 1971-72; 
n «» 93, 1972-73) groups separately. 

Since the objectives of these analyses were to create a limited number 
of distinct but educationally meanlngfia treatment "types," it was first 
necessary to define the variable space within which conblnations of EdExAG 
groups were" to be identified. Although it would have been possible to quan- 
tify differences/similarities among EdExAG groups with respect to all 25 
EdExAG scales, it was decided that the clusters derived would be more Inter- 
pretable and meaningful if the scales used represented measures of the same 
underlying educational construct. Furthermore, if thepe scales described 
what were considered to be major differences in educitlonal approaches, the 
clusters identified would represent a parslmonloita number of types of treat- 
ments which were maximally different with respect to these key treatment 
components. For this reason, the 10 EdExAG scales which indexed some of 
the major ways in which projecc schools were Inclividualizlng instruction 
were selected to be the profile elements describing the approach of each 
EdExAG group. The 10 scales were: Utilization of Objectives, Individual- 
ization in Decision Making, Teacher or Locally Developed Materials, 



Individualization of Instructional Facet Scheduling Characterlstlcst U^e 
of Performance AgreementSt Classroom Group Organlsatlont Teaching Unit Com- 
positlont Completeness of Instructional Packaget and Utilization of Student 
Evaluation. These scales were chosen (1) because they represented ways in 

i 

which sample schools were achieving some degree of individualization; 
(2) because one of the criteria used in selecting schools for participation 
in the project was individualization; and (3) because these scales documented 
some of the major process dimensions that are of current interest in education. 

Nextt each of the EdExAG scale scores was transformed to a deviation 
score by subtracting the scale mean (computed across all EdExAG groups in 
the analysis) . A Q-^type average cross-»products matrix was then computed 
CO provide a multivariate measure of the similarity of each EdExAC group 
with every other group with respect tu the 10 individualization scales. 
Both Nunnally (1967t pp. 372^388) and Overall and Klett (1972t pp. 180- 
239) have discussed such vector*product measures of similarity and have 
noted that they are particularly useful indices since they can be subjected 
to rather **powerful" methods of analysis such as factor analysis. 

Each Q*type average cross-product matrix was subjected to a principal 
factors analysis in which each diagonal value was the average cross-pro* 
duct of an entity (i.e.t EdExAG group) or profile with itself. Four fac* 
tors were extracted in each analysis and rotated to a varimax criterion. 
This number of factors was specified since four factors accounted for 71% 
or more of the trace (sum of the diagonal elements) of each matriXt and 
since a relatively parsimonious solution was desired and since the nuiiiber 
of categories created was expected to be two times the number of factors. 

For purposes of defining cluster membershlpt these Q^type factors 
were viewed as ideal types** of treatments. EdExAG groups were assigned 
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to one of eight clusters on the basis of the rotated factor on which each 
was loaded most highly (in terns of absolute value) and on the basis of 
the sign of that loading. 

In order to interpret the educational characteristics of the groups 
that were formed* the EdExAG scale scores in the original data matrix 
were converted to standard scores and the mean and standard deviation of 
the EdExAG groups included in each cluster t^re computed. A profile for 
each cluster was plotted, each element being equal to the mean standard 
score for all EdExAG groups comprising the cluster. 

Discussion of Cluster Results 

Project LONCSTEP staff who were familiar with the schools and EdExAG 
groups present in the study inspected the mean profiles which had been 
plotted. The most obvious trend in all of these mean profiles was the 
fact that clusters defined as being positively and negatively related to 
the same "ideal treatment type" (i.e., bipolar Q-type factor) tended to 
have profiles which were mirror images of each other. Secondly, the 
highest loading for about one-half of the EdE::AG groups was on the first 
Q-type factor and for this reason, the largest cluster groups were those 
defined by that factor. Third, the treatment approaches defined by the 
first factor were, for both school years, a generally "innovative" group 
and a fairly "traditional" group. Fourth, although there were some sim* 
ilarities of profiles across years, the cood>inations of approaches repre- 
sen ted by the groups defined from the last three Q-type factors were very 
conq>lex and, in general, did not exist for two consecutive school years. 
Last, examination of the profiles by staff familiar wivh each of the 
EdExAG groups from which the clusters were created confirmed that these 
groiq>8 more-or-less reflected the kinds of approaches actually present 
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in the schools. The most valuable result of these procedures, however, 
did not become apparent until the cluster codes were attached to Individual 
students and until student membership in the various combinations of two- 
year "treAtment" paths were examined. 

As noted earlier, the Educational Experience Analysis Guide groups 
Identified during each school year represented the population of different 
kinds of educational approachcj to which participating students were ex- 
posed in a given school year. EdExAG groups were not defined on a longi- 
tudinal basis because the students in the same EdExAG group on 1971-72 
(e.g., Mrs. Jones, Ath grade language arts class) were not necessarily In 
the same EdExAG grcup the next year. For this reason, each individual stu- 
dent's EdExAG group meiia>ershlp was attached to his record. Following the 
cluster analyses described above EdExAG group codes were used to create 
for each student a treatment group code for language arts for 1971-72 and 
for 1972-73. Cross-tabulations of these two codes were then run to show 
the numbers of students following each of the possible longitudinal treat- 
ment paths. 

Examination of these tabulations showed that very few students followed 
a longitudinal coiia>lnatlon of treatments that Involved either "generally 
innovative" ( = 13%) or "generally traditional" ( ; 8%) . Since the 
nature of the other treatment types present during each year was complex 
(though meaningful) and since longitudinal movement of students across years 
was also very complicated, the analysis of the Impact of such complex pat- 
terns of educational treatments over time would tend to produce results 
which would not be amenable to meaningful interpretation. For this reason, 
the typological approach to the analysis of the Project LONGSTEF data base 
was reconsidered. The paper will conclude with a review of the conclusions 
reached. .o. ^ 
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Although the concept of a typology based upon approaches to Individ^ 
uallzation is extremely appealing from both an educational and analytic 
point of view, the results described above more-or-less demonstrated some 
of the shortcomings of such a procedure. First, it is possible that the 
variables used to describe EdEstAG group similarities (i.e., the EdExAG 
scales) were unstable indices of the constructs they purported to measure. 
This is itv^likely, however, since the same project personnel visited the 
same schools in both years and since treatment documentation in 1972-73 
was based upon noting changes from the previous years. It is more likely 
that the 1972'>73 EdExAG data underestimated changes (and were more stable 
than they should have been). 

Second, the treatment "types" identified tended to be very complex 
combinations of component practices and procedures, with the exception of 
the "generally innovative" and "generally traditional" groups developed 
from the first Q-type factor. There are at least two possible explana- 
tions for these results: (1) since the project schools were selected to 
represent a wide range of educational approaches, stable groups could not 
be identified because the sample was both small and very heterogeneous; 
and (2) the clustering method used was not appropriate to these kinds of 
data. 

In order to determine if the clustering results were a function of 
the methodology utilized, the data matrices used in the 1971-72 and 1972-73 
language arts analyses were converted to orthogonal principal component 
scores. Squared distances among EdExAG groups were computed and used to 
cluster the EdExAG groups by means of Ward's (3963) hierarchical grouping 
procedure (Veldman, 1967). Although the mean profiles for a number of clus- 
ters were simil«r (most notably, the "generally traditional" cluster), 

'to 
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there were still substantial differences between the groups created by 
the two techniques* The hierarchical groupst however t were equally com* 
plex and the longitudinal treatment paths of students based on the clusters 
were also complicated* These trends suggested that the cluster results 
obtained were somewhat method dependent* 

This combination of findings also suggested additional explanations 
for our observations* It was quite likely (a) that the kinds of educational 
experiences available to sixth graders were different from those available 
to fifth graders and/or (b) that the practices and procedures themselves 
had changed* Of these twot changes in educational practices over time 
probably provides the most adequate explanation* At an experiential level, 
at least 9 numerous site visitors have noted substantial changes in program 
components over the course of this study-*-innovationt then, appears to be 
as much a process as it is a unique set of school practices* In spite of 
a number of methodological limitations, the cluster results reported in 
this paper more-or-less led to the same conclusion* 

These results also imply that future atten^ts to develop typologies 
of innovative practices that are stable across school years and for different 
samples of schools may not meet with much success* 
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